Method for adaptively encoding motion image based on temporal and spatial complexity and apparatus therefor

ABSTRACT

A method and an apparatus adaptively encode a motion image based on temporal and/or spatial complexity. In the method, encoding is performed with different temporal and spatial resolutions at frame rates based on the temporal and/or spatial complexity of an input image so that image data is stored with high efficiency. The method includes calculating a spatial complexity of input image data, determining a resolution by comparing the calculated spatial complexity with a predetermined threshold, and converting the resolution of the input image data based on the resolution.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. patent applicationSer. No. 10/357,370, filed Feb. 4, 2003. This application claims thebenefit of Korean Application No. 2002-12727, filed Mar. 9, 2002, in theKorean Intellectual Property Office, the disclosure of which isincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a motion image encoding apparatus andmethod, and more particularly, to a method for adaptively encoding amotion image based on temporal and spatial complexity and an apparatustherefor.

2. Description of the Related Art

As digital video recorders (DVRs) and personal video recorders (PVRs)have become popular and widely used, research on image compressiontechniques has been actively performed. However, since an image iscompressed at a fixed resolution in the conventional DVR and PVR withoutconsidering temporal and spatial complexity, the efficiency ofcompression is low.

FIG. 1 is a block diagram of a conventional motion image encodingsystem. First, input image data is divided into 8×8 pixel blocks. Inorder to remove spatial correlation, a discrete cosine transform (DCT)unit 110 performs DCT operations on image data being input in units of8×8 pixel blocks. By quantizing DCT coefficients obtained in the DCTunit 120, a quantization unit (Q) 120 expresses the DCT coefficients bysome representative values such that high-efficiency lossy compressionis performed. A variable length encoding (VLC) unit 130 entropy-codesthe quantized DCT coefficients and outputs an entropy-coded data stream.

An inverse quantization (IQ) unit 140 inverse quantizes the image datathat is quantized in the quantization unit 120. An inverse DCT (IDCT)unit 150 IDCT transforms the image data that is inversely quantized inthe inverse quantization (IQ) unit 140. A frame memory unit 160 storesthe image data that is IDCT transformed in the IDCT unit 150, in unitsof frames. A motion estimation (ME) unit 170 removes temporalcorrelations by using image data of a current frame that is being inputand image data of a previous frame that is stored in the frame memoryunit 160.

For data compression, an MPEG-2 encoder, as shown in FIG. 1, is used inconventional DVRs and PVRs. When input data is not compressed, the datais compressed through the MPEG-2 encoder and the bit stream is stored ina storage medium such as a hard disc drive (HDD) or a digital versatiledisc (DVD). When input image data is a compressed bit stream, in orderto generate an MPEG-2 stream with desired conditions by using a motionimage transcoder shown in FIG. 2, MPEG-2 motion image is decoded andthen, a predetermined scale and format transform is performed. Then,MPEG-2 motion image encoding is performed.

FIG. 2 is a block diagram of a conventional transcoding apparatus. Wheninput image data is a compressed bit stream, image data is decoded by amotion image decoder 220 which comprises a variable length decoding(VLD) unit 222, an inverse quantization unit 224, an IDCT unit 226, aframe memory 228, and a motion compensation (MC) unit 230. Then, inorder to generate an MPEG-2 stream with desired conditions, the motionimage is encoded at a predetermined resolution by using the same MPEG-2encoder as the motion image encoder shown in FIG. 1. This process isreferred to as transcoding. When transcoding is performed, a scale andformat transform unit 240 is used to reduce the scale of the image orconvert the format of the image decoded by the motion image decoder 220,if necessary. Then, the MPEG-2 encoder 260 is used to perform MPEG-2encoding at a predetermined resolution.

Thus, in the conventional MPEG-2 encoding, encoding is always performedwith constant resolution. Accordingly, whether spatial complexity islarge according to the characteristic of an input motion image or not,encoding is always performed with the same resolution. Also, even thougha motion image has little temporal change, a high frame rate of 30 Hz ismaintained, decreasing the efficiency of encoding.

SUMMARY OF THE INVENTION

An aspect of the present invention provides an adaptive motion imageencoding method and apparatus to perform motion image encodingadaptively based on the characteristics of an input motion image so thatencoding efficiency increases.

An aspect of the present invention also provides an adaptive motionimage encoding method and apparatus to maximize an efficiency of storagewhen a motion image is stored in an HDD or DVD by using the MPEG-2compression.

Additional aspects and advantages of the invention will be set forth inpart in the description which follows and, in part, will be obvious fromthe description, or may be learned by practice of the invention.

According to an aspect of the present invention, an adaptive imageencoding method is based on spatial complexity and comprises calculatinga spatial complexity of input image data, determining a resolution bycomparing the calculated spatial complexity with a predeterminedthreshold, and converting the resolution of the input image data basedon the determined resolution.

According to another aspect of the present invention, an adaptive imageencoding apparatus is based on spatial complexity and comprises aspatial complexity calculation unit which calculates a spatialcomplexity of input image data, a resolution determining unit whichdetermines a resolution by comparing the calculated spatial complexitywith a predetermined threshold, and a resolution converting unit whichconverts the resolution of input image data based on the determinedresolution.

According to still another aspect of the present invention, an adaptiveimage encoding method is based on temporal complexity, and comprisescalculating a temporal complexity of input image data, determining aframe rate by comparing the calculated temporal complexity with apredetermined threshold, and converting the frame rate of the inputimage data based on the determined frame rate.

According to still another aspect of the present invention, an adaptivemotion image encoding apparatus is based on temporal complexity andcomprises a temporal complexity calculation unit which calculates atemporal complexity of input image data, a frame rate determining unitwhich determines a frame rate by comparing the calculated temporalcomplexity with a predetermined threshold, and a frame rate convertingunit which converts the frame rate of input image data based on thedetermined frame rate.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages of the invention will bebecome apparent and more readily appreciated from the followingdescription of the preferred embodiments, taken in conjunction with theaccompanying drawings of which:

FIG. 1 is a block diagram of a conventional motion image encodingsystem;

FIG. 2 is a block diagram of a conventional transcoding apparatus;

FIG. 3 is a block diagram of an adaptive motion image encoding apparatusbased on spatial complexity, according to an embodiment of the presentinvention;

FIG. 4 is a block diagram of an adaptive transcoding apparatus based onspatial complexity, according to another embodiment of the presentinvention;

FIG. 5 is a block diagram of an adaptive motion image encoding apparatusbased on temporal complexity, according to an embodiment of the presentinvention; and

FIG. 6 is a block diagram of an adaptive transcoding apparatus based ontemporal complexity, according to another embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Reference will now be made in detail to the present preferredembodiments of the present invention, examples of which are illustratedin the accompanying drawings, wherein like reference numerals refer tothe like elements throughout. The embodiments are described below inorder to explain the present invention by referring to the figures.

The desirability of an image compression technique for the purpose ofstorage lies in whether a given image is compressed with highefficiency. In the conventional DVR and PVR, an image is compressed at apredetermined resolution and then stored.

The present invention solves the problems occurring when an image iscompressed at a predetermined resolution and stored. According to anembodiment of the present invention, considering that temporal andspatial complexity may be large or small, according to thecharacteristics of an input motion image, encoding is performed in unitsof groups of pictures (GOPs) with different spatial and temporalresolutions according to the spatial and temporal complexity of an inputimage, so that high efficiency in storage can be achieved.

After decoding, a GOP having low spatial complexity is expanded throughinterpolation and is then displayed. A GOP having low temporalcomplexity, that is, having little motion, is encoded at a lower framerate so that encoding efficiency increases.

Referring to the accompanying drawings, embodiments of the presentinvention will now be explained.

FIG. 3 is a block diagram of an adaptive motion image encoding apparatusbased on spatial complexity, according to an embodiment of the presentinvention. The motion image encoding apparatus comprises a spatialcomplexity calculation unit 320, a resolution determining unit 340, aresolution converting unit 360, and an encoder unit 380, which areconnected sequentially, that is, in a cascade connection.

The spatial complexity calculation unit 320 obtains dispersion values inunits of 8×8 blocks for input images in units of GOPs and calculates anaverage of the dispersion values. Though the average dispersion value isobtained in units of GOPs in the present embodiment, an averagedispersion value can be obtained selectively for other units.

The resolution determining unit 340 compares the average dispersionvalue, which is calculated in the spatial complexity calculation unit320, with a predetermined threshold and determines a resolution in unitsof GOPs. For example, if an average dispersion value calculated inspatial complexity calculation unit 320 is greater than thepredetermined threshold, MPEG-2 encoding is performed for this GOP witha predetermined original resolution. If the average dispersion valuecalculated in the spatial complexity calculation unit 320 is less thanor equal to the predetermined threshold, resolution of each picture ofthis GOP is lowered, for example, by half in each direction, and thenMPEG-2 encoding is performed for this GOP.

Though in the present embodiment, the spatial complexity is calculatedin units of GOPs and then the resolution is determined, spatialcomplexity may be selectively calculated for other units, for example,in units of sequences and then the resolution is determined. Also,though the resolution is changed for MPEG-2 encoding in the presentembodiment, this can be applied selectively to encoding methods otherthan the MPEG-2.

Here, the predetermined threshold may be determined initiallyconsidering the system environments such as a buffer and memorycapacity, or may be determined arbitrarily by a user according to thetypes of input images.

Also, instead of using a fixed threshold, a threshold may selectively bedetermined adaptively. For example, whether to increase resolution maybe determined by an accumulated average which is obtained fromdispersion values of the previous 8×8 block units. For example, if aunit dispersion value of a GOP is greater than an accumulated average,MPEG-2 encoding is performed for the GOP with an originally desiredresolution. If the unit dispersion value of the GOP is less than orequal to the accumulated average, resolution of each picture of the GOPis lowered, for example, by half in each direction, and then MPEG-2encoding is performed for the GOP. Selectively, the threshold can bedetermined by multiplying an accumulated average by a predeterminedconstant value α. In this case, by appropriately determining a constantvalue α, resolution is lowered, so that the ratio of the GOP to beencoded can be adjusted.

When the above method is performed, encoding can begin only after oneGOP delay, i.e., one GOP delay is required. In order to avoid or reducethe delay, a resolution may be determined by obtaining an averagedispersion value for only the first picture of each GOP. The delay canbe determined appropriately by a user considering a trade-off betweenperformance and delay.

FIG. 4 is a block diagram of an adaptive transcoding apparatus based onspatial complexity according to an embodiment of the present invention.The transcoding apparatus of FIG. 4 comprises a motion image decoder420, a spatial complexity calculation unit 440, a resolution determiningunit 450, a resolution converting unit 460, and an encoder unit 480. Themotion image decoder 420 comprises a variable length decoding (VLD) unit422, an inverse quantization (IQ) unit 424, an IDCT unit 426, a framememory 428, and a motion compensation (MC) unit 430. Here, the motionimage decoder 420 and the encoder unit 480 perform the same functions asin the conventional motion image transcoding apparatus shown in FIG. 2,and therefore, a detailed explanation thereof will be omitted.

When input image data is in a compressed stream, the input image data isdecoded in the VLD unit 422. Regardless of the frame type such as I, B,and P, the spatial complexity calculation unit 440 obtains a dispersionvalue in units of 8×8 blocks in a spatial domain for an input image andcalculates an average in units of GOPs. Though an average dispersionvalue is obtained in units of GOPs in the present embodiment, an averagedispersion value may be obtained selectively for other units, forexample, in units of sequences.

The resolution determining unit 450 compares the average dispersionvalue calculated in the spatial complexity calculation unit 440 with apredetermined threshold and determines the resolution of a current GOP.For example, if an average dispersion value calculated in the spatialcomplexity calculation unit 440 is greater than the predeterminedthreshold, MPEG-2 encoding is performed for this GOP with apredetermined original resolution. If the average dispersion valuecalculated in the spatial complexity calculation unit 440 is less thanthe predetermined threshold, resolution of each picture of this GOP islowered, for example, by half in each direction, and then MPEG-2encoding is performed for this GOP.

Though the spatial complexity is calculated in units of GOPs and thenthe resolution is determined in the present embodiment, the spatialcomplexity may be selectively calculated for other units, for example,in units of sequences and then the resolution is determined. Also,though the resolution is changed for MPEG-2 encoding in the presentembodiment, this can be applied selectively to encoding methods otherthan the MPEG-2.

Here, the predetermined threshold may be determined initiallyconsidering the system environments such as a buffer and memorycapacity, or may be determined arbitrarily by a user according to thetypes of input images.

Also, instead of using a fixed threshold, a threshold may be determinedadaptively. For example, whether to increase the resolution may bedetermined, by an accumulated average which is obtained from dispersionvalues of all the previous 8×8 block units.

When the above method is performed, encoding can begin only after oneGOP delay, i.e., one GOP delay is required. In order to avoid or reducethe delay, the resolution may be determined by obtaining an averagedispersion value for only the first picture of each GOP. The delay canbe determined appropriately by a user considering a trade-off betweenperformance and delay.

The above method is a transcoding method, that is, a bit stream ischanged in order to increase the efficiency of storage. A bit stream maybe stored after transformation with a different bit rate, a differentresolution, or a different standard to satisfy the purpose of theoriginal transcoding.

Also, just like the high efficiency storage by encoding data atdifferent resolutions based on the spatial complexity as describedabove, data may be stored with high efficiency by changing frame ratesbased on temporal complexity.

FIG. 5 is a block diagram of an adaptive motion image encoding apparatusbased on temporal complexity, according to an embodiment of the presentinvention. The motion image encoding apparatus according to the presentinvention comprises a motion activity calculation unit 520, a frame ratedetermining unit 540, a frame rate converting unit 560, and an encoderunit 580.

The motion activity calculation unit 520 obtains motion vectors for allmacro blocks in a GOP, calculates motion activities based on the motionvectors, and calculates an average motion activity of macro blocks. Inthe present embodiment, if a motion vector (MV) of a macro block is(MV1, MV2), the motion activity of the macro block is defined asMV1²+MV2².

The frame rate determining unit 540 compares an average motion activity,which is calculated in the motion activity calculation unit 520, with apredetermined threshold, and determines a frame rate for the GOP andperforms encoding. By appropriately adjusting the threshold, a varietyof frame rate adjusting methods can be carried out. Also, a plurality ofthresholds may be utilized and a frame rate may be adjusted aftercomparing a motion activity with the thresholds. For example, a methodcan be designed such that when determined thresholds are 10 and 20, ifthe motion activity is less than 10, a frame rate of 10 Hz is selected,if the motion activity is less than 20, a frame rate of 20 Hz isselected, and if the motion activity is equal to or greater than 20, aframe rate of 30 Hz is selected. However, in order to prevent problemscaused by a sudden change of a motion (image jerkiness), when a motionactivity is very close to 0, the frame rate of the corresponding GOP maybe adjusted to be relatively low.

The frame rate converting unit 560 adjusts the frame rate of input imagedata according to the frame rate determined in the frame ratedetermining unit 540 and outputs the result to the encoder unit 580.

FIG. 6 is a block diagram of an adaptive transcoding apparatus based ontemporal complexity according to an embodiment of the present invention.The adaptive transcoding apparatus of FIG. 6 comprises a motion imagedecoder 620, a motion activity calculation unit 640, a frame ratedetermining unit 650, a scale transforming unit 660, and an encoder unit680. The motion image decoder 620 comprises a variable length decoding(VLD) unit 622, an inverse quantization (IQ) unit 624, an IDCT unit 626,a frame memory 628, and a motion compensation (MC) unit 630.

When input image data is in a compressed stream, the motion activitycalculation unit 640 calculates motion activities in units of GOPs byusing motion vector information obtained in the decoding step of thecompressed stream. For example, in the embodiment of the presentinvention shown in FIG. 6, the motion activity calculation unit 640receives motion vectors (MVs) output from the variable length decodingunit 622 and calculates a motion activity of each macro block based onthe motion vectors.

In an MPEG-2 stream, a sequence header is followed by a GOP header.Generally, a sequence header is placed once in front of the entiresequence and a GOP header is placed in front of each GOP Image sizeinformation is placed in each sequence header. However, in an actualbroadcast, a sequence header is transmitted for each GOP unit when anMPEG-2 stream is transmitted. Accordingly, if a sequence header is putinto each GOP unit when encoding is performed, decoding can be performedwithout any problem.

The present invention is not limited to the embodiments described above,and it is apparent that variations and modifications by those skilled inthe art can be effected within the spirit and scope of the presentinvention defined in the appended claims.

Also, the present invention may be embodied in a program code, which canbe read by a computer, on a computer readable recording medium. Thecomputer readable recording medium includes all kinds of recordingapparatuses on which computer readable data are stored. The computerreadable recording media includes storage media such as magnetic storagemedia (e.g., ROM's, floppy disks, hard disks, etc.), optically readablemedia (e.g., CD-ROMs, DVDs, etc.) and carrier waves (e.g., transmissionsover the Internet). Also, the computer readable recording media may betransmitted to computer systems connected through a network and canstore and execute a computer readable code in a distributed mode.

According to the present invention as described above, temporal andspatial complexity of an image is calculated in units of GOPs, and a GOPhaving a relatively low temporal and spatial complexity is encoded witha resolution lower than the original resolution, while a GOP having arelatively high temporal and spatial complexity is encoded with theoriginal resolution so that the motion image can be stored in a storagemedium more efficiently.

Although a few preferred embodiments of the present invention have beenshown and described, it would be appreciated by those skilled in the artthat changes may be made in this embodiment without departing from theprinciples and spirit of the invention, the scope of which is defined inthe claims and their equivalents.

1. An adaptive image encoding method based on temporal complexity,comprising: calculating a temporal complexity of input image data;determining a frame rate by comparing the temporal complexity with apredetermined threshold; and determining an outputted frame rate of theinput image data based on the frame rate.
 2. The method of claim 1,wherein the temporal complexity is calculated based on a motion activityof each macro block of the input image data.
 3. The method of claim 2,wherein the motion activity is a size of a motion vector of the macroblock.
 4. The method of claim 3, wherein the motion activity isMV1²+MV2² when the motion vector of the macro block is (MV1, MV2). 5.The method of claim 1, wherein the calculating the temporal complexityand determining the frame rate are performed in units of GOPs of inputimage data.
 6. The method of claim 1, wherein the calculating thetemporal complexity and determining the frame rate are performed inunits of sequences of input image data.
 7. The method of claim 2,wherein if the input image data is encoded image data, the motionactivity is calculated by using a motion vector obtained in a decodingoperation of the input image data.
 8. An adaptive motion image encodingapparatus based on temporal complexity, comprising: a temporalcomplexity calculation unit which calculates a temporal complexity ofinput image data; a frame rate determining unit which determines a framerate by comparing the temporal complexity with a predeterminedthreshold; and a frame rate converting unit which determines anoutputted frame rate of input image data based on the frame rate.
 9. Theapparatus of claim 8, wherein the temporal complexity is calculatedbased on a motion activity of each macro block of the input image data.10. The apparatus of claim 9, wherein the motion activity is a size of amotion vector of the macro block.
 11. The apparatus of claim 10, whereinthe motion activity is MV1²+MV2² when the motion vector of a macro blockis (MV1, MV2).
 12. The apparatus of claim 8, wherein the calculation ofthe temporal complexity in the temporal complexity calculation unit andthe determination of the frame rate in the frame rate determination unitare performed in units of GOPs of input image data.
 13. The apparatus ofclaim 8, wherein the calculation of the temporal complexity in thetemporal complexity calculation unit and the determination of the framerate in the frame rate determination unit are performed in units ofsequences of input image data.
 14. The apparatus of claim 9, wherein, ifthe input image data is encoded image data, the motion activity iscalculated by using a motion vector obtained in a decoding step of theinput image data.
 15. A computer-readable medium having stored thereoncomputer-executable instructions for performing the method of claim 1.